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Introduction 

Problem Definition 



► Given an item(clothing) in the shopping cart the problem 
statement is to suggest items complementary to it which may 
contain garments or accessories which makes a complete set as 
per current fashion. 
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Figure: Existing Recommendation Systems 
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Figure: Visualization of the problem statement 







Mathematical Formulation 

Simplified Formulation 



Mathematical Formulation 


Given an image / containing ‘k’ part-features, we describe the image 
Pi as Pj ■— [p/i,p /2 , ...,p*] where each p,y are textual part-features, 
which are 2-tuples. 
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Mathematical Formulation 


We learn a model from our dataset of fashion images, say P, where P 
:= [P u P z ,...P n ] T . 


Mathematical Formulation 


The task of our recommendation system is, given one or more 
apparel, and corresponding part features p’s as input query, 
recommend garments which can be worn with it/them as a set. 
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Approach 

Flow Diagram 



Figure: Flow Diagram of Proposed Approach 
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Fashion Websites & Ground Truth 

Scraping Fashion Websites 


► Scraped more than 500 images of female fashionstas from 
www.chictopia.com. These images covered an appreciable 
range of street fashion from corporate dressing sense to the 
most casual of the dresses. 
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► Scraped more than 500 images of female fashionstas from 
www.chictopia.com. These images covered an appreciable 
range of street fashion from corporate dressing sense to the 
most casual of the dresses. 

► Created a vocabulary of part features. Manually normalize the 
tags associated with each image. 

► Ended up with a codebook of total of 48 unique categories 
including garments like tops, jeans, etc. and accessories like 
watches, bracelets, etc. and 632 unique items i.e. 
category-description pair. 
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Bipartite Network and Co-occurrence Graph 


► A bipartite graph is formed with dataset images P’s as the first 
partite sets and part-features p,’s as the second partite set. 
There exists an edge between ever part feature and the image in 
which it occurred. 
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Bipartite Network and Co-occurrence Graph 


► A bipartite graph is formed with dataset images P’s as the first 
partite sets and part-features p,’s as the second partite set. 
There exists an edge between ever part feature and the image in 
which it occurred. 

► The bipartite graph is then projected to the set of part features. 

► The projected graph so obtained is a weighted co-occurrence 
graph of the part features. Construction of this graph gives us the 
relation between different garments and accessories which can 
be used together and are complementary to each other. 

► This step helps us learn a correlation and inter-dependence 
between various part features from the dataset. 
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Similarity Measure & Nearest Neighbor 

Similarity Measure 



► The co-occurrence graph falls in a domain where nodes 
represents the objects and edges represents the relations 
between them. We use Simrank to measure the similarity based 
on structural context of the graph. 
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Similarity Measure 



► The co-occurrence graph falls in a domain where nodes 
represents the objects and edges represents the relations 
between them. We use Simrank to measure the similarity based 
on structural context of the graph. 

► Convert the co-occurrence graph into a directed graph where 
each edge between part features p a and Pb in the original graph 
is replaced by two directed edges p a Pb and Pb -» p a both with 
weights equal to the weight of original edge. 
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Similarity Measure 



► The co-occurrence graph falls in a domain where nodes 
represents the objects and edges represents the relations 
between them. We use Simrank to measure the similarity based 
on structural context of the graph. 

► Convert the co-occurrence graph into a directed graph where 
each edge between part features p a and Pb in the original graph 
is replaced by two directed edges p a -» Pb and Pb -» p a both with 
weights equal to the weight of original edge. 

► Compute Simrank between each pair of nodes. 
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Similarity Measure & Nearest Neighbor 

Nearest Neighbor Consensus 


► Given a part-feature p as query we locate the node 

corresponding to that part feature in the co-occurrence graph. 
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Nearest Neighbor Consensus 



► Given a part-feature p as query we locate the node 
corresponding to that part feature in the co-occurrence graph. 

► We find out other nodes which are close to it, i.e. nodes which 
have highest simrank value with this node. 

► The rationale behind this step is that since the graph had edges 
between part features that were used together by fashionistas 
and as the simrank values decrease with increase in node 
distances, the k-nearest-neighbors will be those part features 
which were frequently used with the selected item and are 
contemporary to it. 
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Similarity Measure & Nearest Neighbor 

Nearest Neighbor Consensus 



► Given a part-feature p as query we locate the node 
corresponding to that part feature in the co-occurrence graph. 

► We find out other nodes which are close to it, i.e. nodes which 
have highest simrank value with this node. 

► The rationale behind this step is that since the graph had edges 
between part features that were used together by fashionistas 
and as the simrank values decrease with increase in node 
distances, the k-nearest-neighbors will be those part features 
which were frequently used with the selected item and are 
contemporary to it. 

► We get a list of k part features pi, P 2 , ...pk which are structurally 
close to the input feature and thus they can be recommended for 
the given query part feature. 
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Aggregating Ranked Item Recommendations 

Rank Aggregation 
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► Say we have j part features pi ,p 2 , ...p; as input query, we find out 
individual k-nearest-neighbors for each part feature. 
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► Say we have j part features pi ,p 2 , ...pj as input query, we find out 
individual k-nearest-neighbors for each part feature. 

► we have j ranked lists, each with k members, which are 
recommendation related to each input feature. 

► Assigns a score corresponding to position in which a part feature 
appears within each ranked list. In our case, for each list /', p a ‘ is 
assigned a weight B P J = k * fraction of part features in the list 
appearing below p a . 

► The Broda score of each element B Pa is the the sum of Broda 
scores for that part feature in all the lists. 

► We can recomment the top k elements from this ranked list to 
the user. 
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Experimental Results 

Evaluation Methodology 


► We took 20 images as test set from our dataset. Since each 
image is user tagged, we have labelled ground truth for 
computing the required metrics. 
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Evaluation Methodology 



► We took 20 images as test set from our dataset. Since each 
image is user tagged, we have labelled ground truth for 
computing the required metrics. 

► For each image we took, we used all its part features individually 
as one feature input. We also used various permutations of 2 
part features and 3 part features as input to the recommender 
and compared the recommended part features with the ground 
truth. 

► Then we calculated precision, recall and fl values for 158 sets of 
recommendations. 


Formula 


nmo/ra'nn _ no of matched recommendations 

fjl COISIUI i nQ Q f recommenc j a tj ons 

rmll — no of ma,checi recommendation 
teucw — nQ Qf i tems j n ac t ua i image 
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Experimental Results 

Results 


w 


Out of the 158 recommendation sets that we tested, 53 were 1 part 
feature input, 54 were 2 part feature input and 51 as 3 part feature 
input. For each generated recommendations we calculated the 
precision and recall. 


Table: Precision 


No. of inputs 

Max Precision 

Avg Precision 

1 

1 

0.31 

2 

0.75 

0.31 

3 

0.6 

0.28 


Table: Recall 


No. of inputs 

Max Recall 

Avg Recall 

1 

0.8 

0.23 

2 

1 

0.44 

3 

1 

0.48 

















Experimental Results 

Results I, 


Table: fl score 


No. of inputs 

Max fl 

Min fl 

1 

0.89 

0.13 

2 

0.71 

0.1 

3 

0.67 

0.1 


Precision Recall for 1 item as input 

0.9 
0.8 
0.7 
0.6 
_ 0.5 

ffl 04 

0.3 
0.2 
0.1 
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Figure: Precision-Recall for 1 item input 
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Precision Recall for 2 items as input 



Precision 


Precision Recall for 3 items as input 



Precision 
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Experimental Results 

Manual Evaluation Results 


Table: User rating for recommendation 


Rate(out of 10) 

Frequency 

Cumulative Freq. 

10 

1 

1 

9 

2 

3 

8 

9 

12 

7 

9 

21 

6 

5 

26 

5 

11 

37 

4 

11 

48 

3 

6 

54 

2 

4 

58 

1 

2 

60 








► Features for representation of parts are to be improved by 
incorporating visual features. Inclusion of visual features will also 
include the analysis of features like color, texture, etc. which is 
expected to improve the quality of evaluation. 

► A feedback system can be added to the system as to increase 
edge weights to the features which are shopped together by 
users. This will be a self learning system and incorporate the 
changes in trending fashion all by itself. 
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Thank you! Questions? 



